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This article reports on action research that was undertaken while teaching a mathematics education 
subject to a series of cohorts of final-year primary pre-service teachers at a university in Australia. 
My aim was to increase the understanding of an assessment task, as well as its intrinsic value. Each 
action research cycle involved the introduction of a variation in the assessment task's instructions, a 
process informed by variation theory. The variations were introduced through cycles of planning, 
action, data collection, and reflective analysis related to clarification of the assessment task. The two 
theories (variation theory and action research) proved to be mutually supportive. Cycles of action 
research served well to identify the critical features that needed to be the focus of small variations 
in my presentation of the assessment task. Variation theory provided guidance as to a most 
effective way of highlighting these features, especially with regard to foregrounding varied 
requirements of the task. 


Keywords • pre-service teacher education . open-ended problems . assessment tasks . variation 

theory. action research 


Introduction 

In schools as a teacher, I had always designed assessment tasks that were not only for the 
assessment of learning but were also teaching and learning tasks, and this is a principle that I 
have carried into teacher education. Of course, assessment for learning is not a new idea. It was 
used by Doyle (1983), and later developed extensively by Black and Wiliam (1998a, 1998b). 

The Australian Curriculum Corporation supports the notion of "Assessment for learning". 
Using the definition of the UK's Assessment Reform Group (2002), they defined it as "the 
process of seeking and interpreting evidence for use by learners and their teachers to decide 
where they are in their learning, where they need to go, and how best to get there". Likewise, 
the Education Department in my state, Victoria, explains to teachers that: 

Assessment FOR learning occurs when teachers use inferences about student progress to inform 
their teaching. It is frequent, formal or informal (e.g. quality questioning, anecdotal notes, written 
comments), embedded in teaching, and provides clear and timely feedback that helps students in 
their learning progression. It has a formative use providing evidence that informs, or shapes, 
short term planning for learning. (DEECD, 2013) 
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As a teacher educator, my desire to have pre-service teachers learn through completing 
assessment tasks is continually reinforced by positive student feedback. Many lecturers in the 
faculty talk with our pre-service teachers about formative assessment tasks, and I make a point 
of modelling this principle — and students appreciate this, according to the University's regular 
student evaluations of teaching. Also, the primary pre-service teachers enrolled in the subject 
reported on here are all off campus, completing the subject wholly online, and I know from 
experience that with their busy lives during the final trimester of their course, they tend to focus 
the majority of their attention on completion of assessment tasks more than on learning for its 
own sake. Therefore, it is important to provide them with rich assessment tasks that connect 
their prior experiences with their everyday environment (Lloyd, 2013) for their development as 
teachers in the coming months. 

The focus of our study is based on the research question, "How can we use action research 
to improve the educational design of an assessment-for-learning task, as well as students' 
understanding of the task, in our mathematics education unit?" With this aim in mind, a series 
of changes have been introduced in the unit of study (subject). This paper focuses on the 
variations made in the presentation of the assessment task and the reasons for them. 


Theoretical Framework 

The variation theory of learning offers a way of analysing and describing how learning takes 
place. The theory grew out of phenomenology, with how learners engage with the "objects of 
learning" (Marton, Runesson, & Tsui, 2004, p. 37) being the phenomena that are studied. In 
particular, the focus is on "critical features" of the content: 

... when teaching, we should take as our point of departure what is to be learned (i.e. 
object of learning). For every object of learning and for every learner there are critical 
features that the learners must be able to discern; critical features are critical because the 
learners participating in the study have problems with them, and different learners may 
have different kinds of problems. (Ling & Marton, 2012, p. 9) 

Through comparing and contrasting critical features of the objects of learning with prior 
understandings that have been developed through previous learning experiences, knowledge is 
gradually refined. Seeing phenomena in new ways is at the core of learning. 

In this research, the variations were made in an assessment task, and my aim was to 
increase students' understanding of the task. Thus the "object of learning" was the assessment 
task itself. I was keen to have the students understand the task better and hence to improve the 
outcomes. 

Squares (An Example of an Object of Learning) 

The features of variation theory just described are most easily explained with an example, so I 
will describe my students' (pre-service primary teacher education students) learning about 
squares. 

If I ask a lecture room full of my students what the shape in Figure 1 is called, about 80% of 
them generally say it is a diamond, while relatively few will say it is a square. 
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Figure 1. What is this shape? 

The discussion that follows in the lecture room focuses on critical features of squares: four equal 
straight sides and four right angles. Then I ask the question again, and there is usually general 
agreement that the shape is a square. That is, our defining of a square has challenged the 
students' prior experience of traditionally-oriented squares, and they have realised that the 
orientation does not change either the shape or its name. The learning that has taken place has 
involved the pre-service teachers in "seeing" the phenomena in a new, more discerning way that 
is relevant to their future teaching. 

The focus of the discussion is not about orientation of squares, per se, but this idea is 
"foregrounded" (Bowden & Marton, 1998; Marton & Booth 1997; Marton, Runesson, & Tsui, 
2004). Until that discussion, many of the pre-service teachers have not had their understanding 
about the orientation of squares challenged since their early childhood experience where the 
traditional orientation would probably have been ubiquitous: "Reality and experience of the 
world [were] taken to be one" (Marton & Booth, 1997, p. 148). 

With any such discussions and illustrations, the variations made in an object of learning not 
only focus on what is the same and what remains constant, but also what is different. Marton 
(2006) described the awareness brought about by experiencing such variation between values as 
"contrast", seeing this awareness as being a necessary condition of learning. 

Contrasts between values enable learners to discern values. One can never discern just one value. 

It is the difference that is discerned, and to experience a difference takes at least two values. 

Through contrast, the learner can discern values as well as the dimension of variation in which 

they are values. (Marton, 2006, p. 528) 

In this comparing and contrasting, learners engage with "instances that we have encountered at 
different points in time, at the same time" (Marton, Runesson, & Tsui, 2004, p. 17; italics in 
original). This simultaneous processing of prior experience and new experience is a necessary 
condition for learning (Lo, 2012; Marton, 2006; Marton, Runesson, & Tsui, 2004). 

After more experiences with squares oriented in many different ways, which now fit with 
the students' newly-developed understandings, so are not challenging learning experiences — 
just reinforcement of the concept, I ask if the same figure — the square — is a rectangle. Many 
students call out "No, it's a square!" while a few might say it is a square as well as a rectangle. 
After we look at the critical features of all rectangles, there is usually general agreement on one 
answer. (I will leave interested readers to explore that question with their own students.) 
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The teacher's role 

Any teacher's role is to consider aspects of objects of learning that underpin common 
misunderstandings as well as the salient features of content. Thinking about them involves 
detailed consideration of students' lived worlds as well as the common understandings that 
usually have (or have not) arisen from experience. 

Here, the teachers' experience of student misunderstandings is invaluable. With detailed 
consideration, it is possible to plan learning-focused assessment tasks that create the necessary 
"spaces of learning" (Marton et al., 2004, p. 20) that provide students with opportunities to 
understand both content and pedagogical processes well and to learn from the assessment 
tasks. 

Designing learning-focused assessment tasks for pre-service teachers involves focusing pre¬ 
service teachers' attention on seeing curriculum content and tasks not only through the eyes of a 
learner but also through the eyes of a teacher; so providing them with a range of experiences 
with specific elements of relevant mathematics curriculum content as well as of associated 
pedagogical processes becomes vital. Marton et al. (2004) described the capacity of seeing 
situations that are critical in any profession as "professional seeing" (p. 11), and this is one 
avenue we use to try to develop better understanding of our assessment tasks. 

The development of one particular assessment task is described below to illustrate how 
variation theory has been the focus of my action research. As the unit is offered only once per 
year, each cycle of action research takes one year. That is, variations of the task wording have 
been planned as a result of analysis of student assessment submissions, student feedback, 
analysis of online discussions, and staff discussions; all before changes have been made to the 
written assessment task instructions each year. 

As the trimester progressed and assignments were marked, monitoring of the results has 
been possible. Notes taken by staff at this stage each year, along with consequent staff 
discussions, have helped to inform variations that needed to be made for the following year. 
Thus while students were not able to compare and contrast prior understandings of the task 
that had been developed through previous learning experiences (as in the "Squares" example 
above), the variations we made attended to misunderstandings and errors of previous students. 
In fact, you could say that that we (university staff) were the key learners about the assignment 
task that was the object of learning. 


Methodology 

To improve our presentation of the assessment task to each cohort of students, we used annual 
cycles of action research. Action research is an evolving paradigm (Burns, 2005; Masters, 2000). 
The action research cycle is most frequently traced back to the work of the sociologist Kurt 
Lewin (1946), although elements of like theory can be found in Dewey's writings. (For a 
discussion of pre-cursors see Masters, 2000; but the link with Dewey was later refuted by 
Kemmis, 1960). Lewin pointed out the importance of the participation of practitioners in all 
phases of social change. For Lewin, this new form of participatory research "consisted of 
analysis, fact-finding, conceptualisation, planning, execution, more fact-finding or evaluation; 
and then a repetition of this whole circle of activities; indeed a spiral of such circles" (Sanford, 
1970, p. 4, drawing on a range of Lewin's publications). 

In education in particular, action research is traced back to the teacher-researcher 
movement led by Stenhouse (1975), who envisioned teachers taking an active role in curriculum 
development. Stenhouse proposed that the collection and analysis of data formalises the inquiry 
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processes used by all good teachers. He described action research as a cyclical process of 
planning, action, monitoring, and evaluation that underpins further planning, action, 
monitoring and evaluation, etc. Most importantly, though, he focused on the potential for 
action research to stimulate exploration of everyday practices to enable educators to explore 
historical, cultural, and ideological baggage; and hence to understand better what previously 
had been taken-for-granted. Hence, as an approach to participatory research, educational action 
research is aimed at increasing understanding the social context and limitations of educational 
practice as well as its improvement (Carr & Kemmis, 1986; Kemmis & McTaggart, 1988). 

Action research in education, over the decades, has evolved through various conceptual 
and interpretive "generations" (McKernan, 1996; McTaggart & Garbutcheon-Singh, 1988). 
Typically, now, it involves self-reflective communities of professionals in the simultaneous 
development of educational theory and practice. Teachers and/or teacher educators engage in 
action research to examine their teaching and evidence of their students' learning as a basis for 
making changes, so it is more a means to professional development by an individual or group 
than a way of producing general and generalisable knowledge about teaching and learning. 

Definitions of action research vary, but Sagor (2014) pointed out that however it is defined, 
it has seven steps: 

1. Selecting a focus; 

2. Clarifying theories; 

3. Identifying research questions; 

4. Collecting data; 

5. Analysing data; 

6. Reporting results; and 

7. Taking informed action. 

The second step involves identifying the values, beliefs, and theoretical perspectives that 
researchers hold relating to their focus. As outlined above, it was variation theory that I drew 
on to underpin each of the changes made to an assessment task for pre-service teachers to 
address the research question, "How can we use action research to improve the educational 
design of an assessment-for-learning task, as well as students' understanding of the task, in our 
mathematics education unit?" 

I am not the first researcher to combine variation theory with action research in higher 
education. Akerlind, McKenzie, and Lupton (2014) described a method of curriculum design 
that was based on combining phenomenographic research with the variation theory of learning 
in three stages: (a) identification of concepts needing intensive attention; (b) action research that 
collected data on variations in students' understandings and misunderstandings of those 
concepts; and (c) design of learning activities to address the poorer understandings; using 
variation theory as a guiding framework. This combined method was trialled during a two-year 
project, using physics and law disciplines as case studies. The researchers found that 
"phenomenographic action research served to identify the critical features that needed to be 
highlighted for students [while] Variation Theory provided guidance as to the most effective 
ways of highlighting these features during curriculum design" (p. 1). A key difference between 
the research of Akerlind, McKenzie, and Lupton and mine was that their objects of learning 
were conceptual aspects of the curriculum content, while my research focused on students' 
understanding and performance of an assessment task. 
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The Pre-service Teachers and their Unit of Study 

This report focuses on action research that was undertaken with a mathematics education 
subject that puts emphasis on designing an inclusive program, which is one subject studied in 
the last trimester of the final year of a 4-year Bachelor of Education (Primary). Its students have 
a concurrent extended practicum, so it suits them to study the unit online. Thus the students in 
my unit have no on-campus lectures or tutorials, but there is an online discussion area as well 
as access to substantial online library facilities. Earlier in their course, the pre-service teachers 
have all passed at least one unit of Mathematics that was taught by the Faculty of Science and 
Technology as well as two other units of Mathematics Education. A few have studied more 
Mathematics as elective units. 

The "Problem Pictures" Task 

In this unit, the pre-service teachers are assessed partly on their submission of a multimedia 
resource that focuses on mathematics curriculum. One task is to take a photograph of the 
everyday environment and to write three open-ended mathematics questions (often called 
"open questions") that are suitable for different grade levels (see Figure 3). It is this "Problem 
Pictures" task that has been the focus of my recent research. The image and questions below 
have been presented to pre-service teachers as an example of possible open-ended questions 
that can be designed based on an image. 



Photograph with permission of Ben West 

Question 1, for junior primary children: Describe and sort the shapes you see in the photo. (Teacher 
note: depending of the grade, the focus might be number of faces, edges, vertices or 2D and 3D 
shapes). 
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Question 2, for middle primary children: What time of day might this photo be taken? Explain and 
justify your reasoning. 

Question 3, for upper primary children. Design a grid for the photo. Select an object in the photo and 
state its coordinates to describe the location of the object. 


Figure 3. A Problem Picture with three open-ended questions. 

In their prior mathematics education units, the pre-service teachers have met the idea of using 
open-ended questions (see, for example, Mousley, Sullivan & Zevenbergen, 2004; Sullivan & 
Lilburn, 1998) in mathematics classrooms, but they have had little practice in constructing these. 
The "Problem Pictures" learning and assessment task that is the focus of this paper reinforces 
and assesses pre-service teachers' abilities to construct suitable open-ended mathematics 
questions. We, the teaching staff in this unit, believe that learning about open-ended 
mathematics questions is an essential aspect of being able to design an inclusive program. 

The most important characteristic of open-ended questions is that they have more than one 
possible correct solution. In mathematics classrooms, they are useful for fully engaging children 
with a range of abilities and hence being inclusive as well as using time productively. Take, for 
example, "A rectangle has an area of 24cm 2 . What might its dimensions be?" Those children 
who are generally less proficient in mathematics (perhaps using an enabling resource such as 
some squared paper) might find only one or a few answers to this question during the same 
time that others will find many correct solutions — perhaps using a logical pattern. With well- 
designed open-ended tasks, all children can make a start and all will have the opportunity to 
extend their knowledge. The whole class is working on a common learning task so all are able 
to make contributions to, and to learn further from, a class discussion about their responses and 
the mathematical concept being studied. 

In Australia, where mathematics textbooks are generally not used heavily and curriculum 
guidelines can be relatively concise, primary teachers need to be able to create suitable learning 
tasks — and the use of open-ended questions is one aspect of designing an inclusive program. 
Hence learning to design suitable mathematical questions and tasks that allow for open entry 
(all students to start) and open exit (potential for extension and generalisation) is a standard 
practice in many primary teacher education programs. We aim to reinforce as well as to assess 
pre-service teachers' understanding of and ability to apply this skill in our unit of study. 

For the assessment task, the pre-service teachers are asked to create a set of open-ended 
questions that are linked to state curriculum content, that arise from a photograph that they 
have taken. The steps involve taking and submitting a photograph, along with three open 
mathematics questions that use different mathematical concepts for three differing grade levels. 

We do not talk with the students about variation theory; but its principles include that in 
any assessment task where we want to find out what has been learned, we must introduce 
variation between the way something is learned and the way it is assessed (Marton, 2006; 
Smedslund, 1953) to avoid rote learning — this time into technology use. Marton et al. (2004) 
advised that the main aim of any learning task must be for students to "see" and understand 
differently: "We try to achieve our aims, not in relation to the situation in an objective sense but 
in relation to how we see it" (p. 5). 

For this task, over the past eight years, eight cohorts of pre-service teachers (n ~ 2,800) have 
submitted about 9,800 photographs and posed a total of about 29,400 open-ended mathematical 
problems, questions, or tasks that were based on their problem pictures. The initial aims for this 
assessment task were (a) for the pre-service teachers to notice mathematics beyond the 
classroom walls; and (b) to assess students' ability to write open questions. However, cycles of 
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action research over this time — consciously informed by variation theory — have resulted in 
important features of the task being better scaffolded and more productive. The progressive 
variations introduced in the different cycles of action research are described later in this paper. 

Data Collection and Analysis 

Over an eight year period, the photographs that the students have submitted each time the unit 
has been taught annually (which we called "Problem Pictures") as well as the attendant 
mathematical questions that they posed have been analysed. Over time, my idea has been to 
increase the variety of environmental contexts of the photographs (as evidencing growing 
awareness of mathematics in everyday contexts) as well as the types of problems posed and the 
range of mathematical foci and links between these (evidence of knowledge of the primary 
curriculum and understanding of linked mathematical concepts). 

Data were also generated when the pre-service teachers were asked to write about what 
they thought they had learned from the task, and how. Specific questions that they were asked 
to answer in writing included "In what ways (if any) did this task help you learn more about 
teaching mathematics?" and "What, if any, were the benefits of completing the Problem Pictures 
task?" 

While I am the Unit Chair, some lecturers from the other campuses of the university also 
teach the unit, and discuss potential developments of the unit via email; and a professional 
colleague in Canada has used adaptations of the task and engaged in discussions with me. 
Points made by the pre-service teachers, recorded in the unit's online discussion space, also 
became data — along with any relevant comments from the formal, periodic university Student 
Evaluation of Units and Teachers. We have also received feedback and suggestions from other 
academics when describing and sharing data via academic conferences and journal articles 
(Bragg & Nicol, 2008, 2011, 2013, 2015; Nicol & Bragg, 2009). Data analysis at each of the eight 
iterations of the assessment task took the form of document analysis (McKernan, 1996) using 
the pre-service teachers' work samples, online posts, emails, and survey responses; all 
supported by self-reflection on the verbal input of colleagues and notes taken by staff when 
marking. 

Tests of pre-service teachers' learning outcomes were not possible within the project design, 
which placed a greater priority on the educational development and task improvement over 
different cohorts of students. The use of a control group of students was also not possible, as 
comparison of any current year student outcomes with the outcomes of the previous year's 
students would not be valid. Thus my aims in this paper are not about providing evidence that 
the variations in practice worked — at least not in a quantifiable and replicable way — but to 
describe the main foci for the action research with its application of variation theory. 


Findings of the Research 

Posing open-ended problems is challenging for many in-service and pre-service teachers. It was 
evident from my continued engagement with this task that support for creating, posing, and 
adapting open-ended questions was necessary to develop students' capabilities as mathematics 
instructors. Through an annual cycle of action research variations to the assessment task have 
been incorporated. Looking back over the series of changes made sequentially, there were three 
critical features of the task that were varied: (a) clarification of the task, including choice of 
pictures and posing of open-ended questions; (b) more focus on inclusive teaching, which 
brought in varied grade levels as well as within-class differences; and (c) new opportunities for 
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seeing the task not just from the perspective of a student but also through the eyes of peers as 
well as a child. Details of these three themes are outlined below. 

Clarification of the Task: Choosing Photos and Problems 

At first students seemed to have trouble identifying aspects of the environment that would 
make a photograph "interactively" mathematical. Photographs in the initial submissions were 
largely decorative. For example, one student submitted a picture of a tram (a feature of 
Melbourne transport and a tourist attraction) and asked questions to do with time, distance, 
and speed, but the mathematics was not illustrated in the photograph as it would have been in, 
for instance, a photograph of the tram's timetable. Illustrative photographs did not meet one 
initial aim of the task: for the pre-service teachers to become more discerning about the 
mathematics embedded in the everyday environments of children — with potential for 
subsequent recognition of critical features of environmental factors for their mathematical 
potential. 

Analysis of assignments showed that I needed to be more explicit about the mathematics 
"residing" in the photograph, meaning the pupils would be required to examine the content of 
the photograph in order to complete its accompanying problems. Such a problem picture was 
deemed "interactive". In the spirit of variation theory, some contrasting photographs were 
described as "illustrative" (Bragg & Nicol, 2011, p. 7). It was not long before "residing", 
"interactive", and "illustrative" became terms that the students started to use with confidence in 
the unit's online discussion area — and their choice of interactive pictures improved greatly. 

Now a better description of the task is not only in the assignment description, but also in 
the unit's discussion space. The following is one message posted by me early in each trimester: 

Hi All. This is just a little reminder to start exploring your environment for potential 
mathematical opportunities for your Problem Pictures task. Take a minute right now to look 
around you. Can you see something that might be mathematically intriguing? Take a photo, take 
a few photos. You might not realise the potential right away but can come back to your shots later 
for inspiration. Remember that the mathematics needs to reside in the photos so that students 
will use the images interactively to answer the open questions. The images are not just pictures 
that illustrate what your problems are about. [Administrative notes including a reminder about 
the due date.] Have a wonderful time noticing the mathematics around you. 

In the discussion space, students ask questions and make helpful suggestions about this 
requirement. A recent post was: 

A good way to test if your question is illustrative or interactive is to remove the photo and see if 
you can still answer the question. If you can, then the photo is illustrative. (Discussion post) 

Also early in the trimester, I now model some suitable photographs and pertinent open-ended 
questions during several online non-compulsory activities that scaffold the students' 
understanding of the task. In the discussion space, I present some sample photographs and 
attached open questions, both to highlight the features of a problem picture and to show how 
problem solvers need to look closely at the picture to answer the questions. This is supported by 
a description of what makes the picture interactive and the problems appropriate. (The picture 
and problems are selected from a collection made over the years, and they are ideal for 
promoting noticing of quite a range of mathematical concepts.) 

The next step in the current action research was to post further photographs without 
problems, asking the pre-service teachers to write suitable open-ended mathematical problems 
that require interaction with the pictures. Feedback from academic staff and peers is 
encouraged, and my online comments intentionally include how the problem pictures could be 
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adapted for (a) different grade levels through varying the grade-level content of the problems 
whilst maintaining the same mathematical learning intention; and (b) varying the mathematical 
learning intention (concept and/or skill) while holding the grade level constant. As we discuss 
what has been changed compared with what has been held constant, and with what 
implications, the pre-service teachers are learning about how to adapt tasks to suit specific 
student groups via variance and invariance — although variation theory itself is not mentioned 
nor part of the course content. 

The final scaffolding step related to choice of pictures and creation of open-ended 
mathematics problems has been to post an open-ended problem for one particular photograph 
and to invite students to share other photographs or descriptions of settings where this problem 
might also be asked. 

We have found that this set of structured but non-compulsory activities based on the 
foregrounding of critical features of the assessment task has greatly enriched the learning, as 
evidenced by most students now submitting varied and suitable photographs together with a 
good range of appropriate open-ended mathematical problems. 

Inclusive Teaching 

It is important for pre-service teachers to learn to be inclusive in their teaching. In this unit of 
study, as well as other units, the students consider the need to cater for gender, social class, 
race, second-language, ethnic, and special education needs; but for this assignment I focus on 
use of open-ended questions as a way of catering for a range of ability and achievement levels, 
including children who are well above or well below national expectations. Here, I am trying to 
offer the pre-service teachers options other than the ubiquitous so-called "ability" grouping that 
risks disaffection, polarisation, and the construction of failure (Boaler, Wiliam & Brown, 1998; 
Dahllof, 1971) as well as its reproduction of class-based inequities (Boaler, 1997). Turner 
Harrison (2003) found that teachers are able to give examples of open-ended questions and 
tasks that engage the full range of children successfully. She noted that it is not only the 
opportunity for all to have success but also the nature of that success that is important: "the 
positive sense of the students feeling in control" (p. 77). 

In the first years that the Problem Pictures task was used, students showed little awareness 
of the need to consider inclusivity. Many of their questions tended to be closed or to have a very 
limited range of responses, and their written rationales for choosing their questions rarely 
included notions of catering for different achievement levels. To put more emphasis on this, I 
varied the assignment instructions by including more specific directions for the rationale, as 
follows: 

"The rationale (400 words) should address the following questions (included relevant citations): 

How are these problems inclusive of a range of children? 

How would you employ problem pictures in the classroom? 

What are the benefits of problem-pictures/open-ended questions in the classroom?" 

The set readings that were added to support students' writing about these points emphasise 
inclusive teaching of mathematics. Further, the unit already had a set of videoed lectures, 
including one titled Teaching Mathematics to Classes of Diverse Interests and Backgrounds (Sullivan, 
2005), that were used in a different assessment task, so the "inclusive teaching" aspect of these is 
already foregrounded by lecturers in the discussion space. 
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These changes seem to have been effective variations. There is certainly more awareness 
shown of inclusive aspects in the unit discussion space as well as in the quality of open-ended 
questions and the pre-service teachers' written rationales. 

Professional Seeing 

Variation theory suggests that pre-service teachers need to develop a dual in situ perspective of 
any object of learning from their experience as (a) a learner; as well as (b) a teacher (Lingefjard, 
2011). We have now set up part of the set task to support pre-service teachers' looking at their 
problem pictures and questions with a critical, professional eye by looking through the eyes of 
others: first their peers and then by sample children. 

Prior to submission of the problem pictures and questions for grading, the pre-service 
student teachers now undertake and report on two phases of trialling. First, they arrange — 
online or with a group of face-to-face peers (perhaps in their practicum schools) — for detailed 
feedback from fellow students, which might result in several iterations as they engage with 
several lots of critique. The peers gain insights from experiencing their peers' problems as 
learners, making further suggestions about the pictures' mathematical potential, and discussing 
the wording and potential solution processes of the open-ended questions. 

It is often easier for other people to pick up poorly worded questions, typographical errors, 
and potential pedagogical problems than it is for authors of the tasks. The instructions now 
require that the peers give some answers (plural) for the open-ended questions, state what they 
perceived to be the learning intention, and suggest ways to enhance the effectiveness of the 
problem picture in achieving the intended learning. Overall, we have found that the pre-service 
teachers take this peer tutoring seriously, and student feedback has shown that this 
implementation of change has been highly appreciated. 

A subsequent stage of "looking through the eyes of others" that was introduced was for 
students to trial their problem pictures face-to-face with at least three primary-school children 
from the grades for which their questions are designed. They collect the children's written 
responses as work samples, and complete a written reflection in response to four prompts: 

1. Did the student answer my problem picture as expected? If not, why not? State 
your evidence. 

2. What strengths or weaknesses of the students' mathematical understanding were 
highlighted through this question? 

3. Did the problem address the mathematical intent of the question? If yes, how? If no, 
why not? 

4. How might I modify/rewrite my problem picture in light of the students' responses 
and my reflective process, to address the mathematical intent of the question? 

Trialling with children has alerted students to potential problems and has often resulted in 
rewording of questions, as shown in the following written reflection: 

Reflection on child's response: 

My original question was 'How many shapes can you see in the picture? Draw and name them, 
and state how many edges and corners they have.' 

"Lily" had a little bit of trouble understanding what the question was asking and was going to 
count all of the shapes she could see in the photo, until I explained that I was looking for the 
number of different shapes she could see and not every single shape in the photo. Once this was 
explained to her, she drew a triangle, rectangle, and square and correctly labelled them. 



131 




Action research on the application of variation theory 


Bragg 


The main issue with the question was not the question idea itself but the wording I had originally 
used. I have modified it so that it clearly states what is required in the answer. The rephrased 
question is: 

How many different shapes can you see in the picture? Illustrate your answer by drawing and 
naming them, and label how many edges and corners each one has. 

Thus the pre-service teachers are now experiencing their problems through two sets of eyes 
(peers' and pupils') before adapting their assignment content accordingly ready for submission. 
Trialling and reporting on the results have increased their workload, but the submitted 
assignments are now of a very high standard and the open-ended questions are usually well 
worded, error free, and more appropriate for the grade levels nominated. 


One Ongoing Variation of the Task 

According to feedback from fellow academics (co-teachers) and the most recent round of 
analysis of research data collected, a small number of pre-service teachers still continue to find 
it difficult to assess whether their problems are open-ended or closed. One strategy to help such 
students has been trialled this trimester with a variation of the task description: getting the pre¬ 
service teachers to write, after each question, three possible responses. While this inclusion has 
assisted many, radically reducing the number of closed problems submitted, a few pre-service 
teachers in the current cohort presented a closed question with (say) one correct and two 
incorrect responses. Perhaps they understood the requirement as just writing three answers 
children might offer, whether correct or not. So the next iteration of the task description (for the 
present cohort undertaking the unit at the time of writing) will ask for "three possible correct 
responses". This is likely to bring about a better understanding of the nature of open-ended 
mathematics questions. 

In summary, the variations made to the assessment task instructions are detailed in Table 1. 


Table 1 

Variations Implemented to the Assessment Task through each Action Research Cycle 


Variation 


1st 

variation 


2nd 

variation 


Issue Identified 


Questions were 
predominately closed. 
Analysis of problem 
pictures indicated that 
approximately 39% were 
illustrative and 61 % 
were interactive. 

The mathematical intent 
of the question was not 
easily understood and 
the phrasing was poor. 


More focused support 
needed for creating 


Major Variation Implemented in the Assessment Task 
Instructions 

"The problems must be open-ended. After each 
question write 3 possible answers to your questions." 
"Ensure the mathematics in the questions resides in 
the photos. Questions should link directly to the 
content of the photo so the photo is interactive rather 
than illustrative." 

"Initially trial your questions with your peers. Ask 
your peers to provide you with some possible 
answers for the open-ended questions, state what 
they perceived to be the learning intention, and 
suggest ways to enhance the effectiveness of the 
problem picture in achieving the intended learning. 
Based on their feedback, you may wish to rephrase 
your problem questions" 

"Support material modelling Problem Pictures is 
located online. This material is a work in progress and 
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3rd 

variation 


4th 

variation 


5th 

variation 


6th 

variation 


open-ended questions. 

Little variation in the 
mathematical concepts 
represented in the 
questions. 

Many of the questions 
were not engaging or the 
PSTs had difficulty 
forming suitable 
questions. 


The content of some 
photos was dull and 
easily found in the 
classroom - therefore 
negating the need for a 
photo. 


Support for considering 
creative contexts for 
open-ended questions. 


The wording of many of 
the problems were 
clumsy or confusing. 


Questions were not 
inclusive of a range of 


will grow with more examples of Problem Pictures 
shared amongst academic staff and your cohort." 
"Each of the content strands of the National 
Curriculum document must be represented in the 
range of questions." 

"We have created an online space for you to examine 
a preselected photo and invite you to pose or refine 
questions to share with your peers. Academic staff 
and your peers will provide feedback on your 
questions. This is a supportive environment created to 
assist you in honing your open-ended problem posing 
skills." 

"Do not create photos of objects you can find easily in 
a classroom, e.g. dice, unifix blocks, counters, number 
charts, standard clocks. In the real setting of the 
classroom you would not use a photo of unifix blocks 
when you have them next to you. This includes items 
that are easily portable such catalogues, flyers, 
portable menus which you would bring into the 
classroom rather than take a photo of it. Photos of a 
school or classroom are also not suitable for this task." 
"Considering creative contexts for your problem 
pictures is a unique way to engage your students in 
mathematics. In the online discussion forum we have 
placed one open-ended problem and a matching 
photo. We invite you to share a photo or describe a 
context to match this problem in an interactive 
manner. Be imaginative!" 

"After trialing your questions with your peers, you 
must trial every question with at least one 
appropriately-aged child. Give the child the photo 
and open-ended question. Ask the child to respond to 
your question in written or oral form. 

Reflect on the children's responses to each question. 
Include relevant citations to readings you have 
sourced yourself outside the unit materials. You need 
to address the reflective prompts below. Ask yourself: 

• Did the child answer my problem picture as 
expected? If no, why not? 

• What strengths or weaknesses of the child's 
mathematical understanding were highlighted 
through this question? 

• Did the question address the mathematical intent of 
the question? If yes, how? If no, why not? 

• How might I modify my question in light of the 
child's response?" 

Write an enabling prompt and an extending prompt 
for each question for your grade. These prompts 
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children and some 

address the same mathematical concept as the 

children found the 

original problem. 

original questions 

The rationale (400 words) should address the 

difficult to access or too 

following questions (included relevant citations): 

simple. 

• How are these problems inclusive of a range of 
children? 


• How would you employ problem pictures in the 
classroom? 


• What are the benefits of problem-pictures/open- 
ended questions in the classroom? 


Conclusion 

In this study, the systematic annual cycles of analysis, planning, implementation, and further 
data collection of action research served well to identify the critical features that needed to be 
the focus of small changes in the assignment. Variation theory provided guidance as to ways of 
highlighting critical features of the assessment task features, especially with regard to choosing 
pictures and posing appropriate problems, thinking about inclusiveness, and seeing 
mathematics questions through the eyes of a teacher, of peers, and of potential pupils. 

Through using cycles of action research where different aspects of the assessment task have 
been varied over time, my understanding of this task as well as my grasp of its potential for 
enabling stronger "professional seeing" by the pre-service teachers have been developed. So not 
only are students coming to see open-ended questions in mathematics differently, but I am 
seeing this task and the process of assessment through new eyes. Thus, I believe that employing 
action research, with a focus on variations to the task, in the manner described above offers 
teacher educators greater potential to gain insights into their practice as well as to recognise the 
benefits that an iterative process of variation to assessment tasks has for their pre-service 
teachers. The changes applied to the task over the years have made it a more comprehensive 
assignment task that is also now better understood by the pre-service teachers. The teaching 
staff have noticed an improvement in the quality of problem picture submissions and the task is 
the subject of many positive comments in the university's formal evaluation of the unit and its 
teaching. 
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